AppScale: Open-Source Platform-As-A-Service

AppScale is a scalable, distributed, and fault tolerant cloud runtime system that we have developed at the University of California, Santa Barbara as part of our research into the next generation of programming systems [5, 3]. In particular, AppScale is a cloud platform, i.e. a platform-as-a-service (PaaS) cloud fabric, that executes over cluster resources. The cluster resources underlying AppScale can be managed with or without virtualization (e.g. Xen, KVM) or via popular cloud infrastructures including Amazon EC2 [6] and Eucalyptus [8]. The AppScale platform virtualizes, abstracts, and multiplexes cloud and system services across multiple applications, enabling write-one, run-anywhere (WORA) program development for the cloud. In addition to simplifying application development and deployment using cloud systems and modern distributed computing technologies, AppScale brings popular public cloud fabrics to “on-premise”, or private, clusters. To enable this, we emulate key cloud layers from the commercial sector – (i) to engender a user community, (ii) to gain access to and to study real applications, and (iii) to investigate the potential implementations of, and extensions to, public cloud systems using open-source technologies. The first API we have chosen to emulate is that of Google App Engine [11]. Google App Engine is a cloud platform that provides scalable implementations of technologies important for web services (messaging, key-value data storage, multi-tasking, web server support, elasticity, and resource management, among others). Using Google App Engine developers debug and test their programs using an open-source software development kit (SDK) provided by Google that implements non-scalable versions of the APIs. Developers then upload their code and data to Google clusters, and can then use Google resources and services “pay-per-use”, on a free or low-cost rental basis. AppScale implements the (open) Google App Engine APIs such that applications that execute on Google App Engine, also execute on AppScale without modification using private cluster resources or public cloud infrastructures. Our API and service implementations target scale, distribution, fault tolerance, high-performance, and high availability. We leverage mature open-source technologies to the greatest degree possible to enable this. AppScale implements multiple language runtimes (Java, Python, Ruby) as application frontends and a wide range of open source database technologies (key-value or relational) as options for its internal system-wide datastore. AppScale is not a replacement for Google App Engine or any other public cloud technology. Instead, AppScale is a robust and extensive distributed system that provisions the resources it is allocated scalably across multiple applications. The AppScale platform exports services and APIs in addition to those provided by Google App Engine. These technologies are important for application domains beyond those of web services, e.g. data analytics and data and computationally intensive applications. AppScale exports these technologies as services, i.e. AppScale “service-izes” libraries, tools, and software packages, including MapReduce, X10, R, and MPI, and, as it does for the Google App Engine APIs, AppScale provides automatic configuration, deployment, and distribution for them, as well as support for their elasticity, load balancing, and fault tolerance. Since AppScale provides a software layer between applications and the implementations of cloud APIs and services, we are also able to employ existing cloud services for the implementations. As such, AppScale provides a hybrid cloud platform – a programming system through which an application can access services from different clouds (a mix of public and private) at different times or concurrently. Such support can be used by developers to move data between clouds, e.g., for disaster recovery, fault tolerance, data backups, to reduce public cloud costs (to use lower-cost alternatives), and to “fall out” from a private cloud with limited resources to a public cloud on-demand.